Serveur d'exploration sur la recherche en informatique en Lorraine

Attention, ce site est en cours de développement !
Attention, site généré par des moyens informatiques à partir de corpus bruts.
Les informations ne sont donc pas validées.

Projection-based demixing of spatial audio

Identifieur interne : 000033 ( Main/Exploration ); précédent : 000032; suivant : 000034

Projection-based demixing of spatial audio

Auteurs : Derry Fitzgerald [Irlande (pays)] ; Antoine Liutkus [France] ; Roland Badeau [France]

Source :

RBID : Hal:hal-01260588

English descriptors

Abstract

We propose a method to unmix multichannel audio signals into their different constitutive spatial objects. To achievethis, we characterize an audio object through both a spatial and a spectro-temporal modelling. The particularity of the spatialmodel we pick is that it neither assumes an object has only one underlying source point, nor does it attempt to model the complexroom acoustics. Instead, it focuses on a listener perspective, and takes each object as the superposition of many contributionswith different incoming directions and inter-channel delays. Our spectro-temporal probabilistic model is based on the recentlyproposed α-harmonisable processes, which are adequate for signals with large dynamics, such as audio. Then, the mainoriginality of this work is to provide a new way to estimate and exploit inter-channel dependences of an object for the purposeof demixing. In the Gaussian α = 2 case, previous research focused on covariance structures. This approach is no longervalid for α < 2 where covariances are not defined. Instead, we show how simple linear combinations of the mixture channelscan be used to learn the model parameters, and the method we propose consists in pooling the estimates based on manyprojections to correctly account for the original multichannel audio. Intuitively, each such downmix of the mixture provides anew perspective where some objects are cancelled or enhanced. Finally, we also explain how to recover the different spatial audioobjects when all parameters have been computed. Performance of the method is illustrated on the separation of stereophonic musicsignals. Index Terms—source separation, probabilistic models, non-negative matrix factorization, musical source separation

Url:


Affiliations:


Links toward previous steps (curation, corpus...)


Le document en format XML

<record>
<TEI>
<teiHeader>
<fileDesc>
<titleStmt>
<title xml:lang="en">Projection-based demixing of spatial audio</title>
<author>
<name sortKey="Fitzgerald, Derry" sort="Fitzgerald, Derry" uniqKey="Fitzgerald D" first="Derry" last="Fitzgerald">Derry Fitzgerald</name>
<affiliation wicri:level="1">
<hal:affiliation type="laboratory" xml:id="struct-246988" status="VALID">
<orgName>NIMBUS Centre [Cork]</orgName>
<orgName type="acronym">NIMBUS</orgName>
<desc>
<address>
<addrLine>Cork Institute of Technology Campus, Bishopstown, Cork</addrLine>
<country key="IE"></country>
</address>
<ref type="url">http://nimbus.cit.ie</ref>
</desc>
<listRelation>
<relation active="#struct-313560" type="direct"></relation>
</listRelation>
<tutelles>
<tutelle active="#struct-313560" type="direct">
<org type="institution" xml:id="struct-313560" status="INCOMING">
<orgName>Cork Institute of Technology</orgName>
<desc>
<address>
<country key="FR"></country>
</address>
</desc>
</org>
</tutelle>
</tutelles>
</hal:affiliation>
<country>Irlande (pays)</country>
</affiliation>
</author>
<author>
<name sortKey="Liutkus, Antoine" sort="Liutkus, Antoine" uniqKey="Liutkus A" first="Antoine" last="Liutkus">Antoine Liutkus</name>
<affiliation wicri:level="1">
<hal:affiliation type="researchteam" xml:id="struct-420403" status="VALID">
<idno type="RNSR">201421147E</idno>
<orgName>Speech Modeling for Facilitating Oral-Based Communication</orgName>
<orgName type="acronym">MULTISPEECH</orgName>
<desc>
<address>
<country key="FR"></country>
</address>
<ref type="url">http://www.inria.fr/equipes/multispeech</ref>
</desc>
<listRelation>
<relation active="#struct-129671" type="direct"></relation>
<relation active="#struct-300009" type="indirect"></relation>
<relation active="#struct-423086" type="direct"></relation>
<relation active="#struct-206040" type="indirect"></relation>
<relation active="#struct-413289" type="indirect"></relation>
<relation name="UMR7503" active="#struct-441569" type="indirect"></relation>
</listRelation>
<tutelles>
<tutelle active="#struct-129671" type="direct">
<org type="laboratory" xml:id="struct-129671" status="VALID">
<idno type="RNSR">198618246Y</idno>
<orgName>INRIA Nancy - Grand Est</orgName>
<desc>
<address>
<addrLine>615 rue du Jardin Botanique 54600 Villers-lès-Nancy</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.inria.fr/nancy</ref>
</desc>
<listRelation>
<relation active="#struct-300009" type="direct"></relation>
</listRelation>
</org>
</tutelle>
<tutelle active="#struct-300009" type="indirect">
<org type="institution" xml:id="struct-300009" status="VALID">
<orgName>Institut National de Recherche en Informatique et en Automatique</orgName>
<orgName type="acronym">Inria</orgName>
<desc>
<address>
<addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.inria.fr/en/</ref>
</desc>
</org>
</tutelle>
<tutelle active="#struct-423086" type="direct">
<org type="department" xml:id="struct-423086" status="VALID">
<orgName>Department of Natural Language Processing & Knowledge Discovery</orgName>
<orgName type="acronym">LORIA - NLPKD</orgName>
<desc>
<address>
<country key="FR"></country>
</address>
<ref type="url">http://www.loria.fr/la-recherche-en/departements/Knowledge-and-Language-Management</ref>
</desc>
<listRelation>
<relation active="#struct-206040" type="direct"></relation>
<relation active="#struct-300009" type="indirect"></relation>
<relation active="#struct-413289" type="indirect"></relation>
<relation name="UMR7503" active="#struct-441569" type="indirect"></relation>
</listRelation>
</org>
</tutelle>
<tutelle active="#struct-206040" type="indirect">
<org type="laboratory" xml:id="struct-206040" status="VALID">
<idno type="IdRef">067077927</idno>
<idno type="RNSR">198912571S</idno>
<idno type="IdUnivLorraine">[UL]RSI--</idno>
<orgName>Laboratoire Lorrain de Recherche en Informatique et ses Applications</orgName>
<orgName type="acronym">LORIA</orgName>
<date type="start">2012-01-01</date>
<desc>
<address>
<addrLine>Campus Scientifique BP 239 54506 Vandoeuvre-lès-Nancy Cedex</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.loria.fr</ref>
</desc>
<listRelation>
<relation active="#struct-300009" type="direct"></relation>
<relation active="#struct-413289" type="direct"></relation>
<relation name="UMR7503" active="#struct-441569" type="direct"></relation>
</listRelation>
</org>
</tutelle>
<tutelle active="#struct-413289" type="indirect">
<org type="institution" xml:id="struct-413289" status="VALID">
<idno type="IdRef">157040569</idno>
<idno type="IdUnivLorraine">[UL]100--</idno>
<orgName>Université de Lorraine</orgName>
<orgName type="acronym">UL</orgName>
<date type="start">2012-01-01</date>
<desc>
<address>
<addrLine>34 cours Léopold - CS 25233 - 54052 Nancy cedex</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.univ-lorraine.fr/</ref>
</desc>
</org>
</tutelle>
<tutelle name="UMR7503" active="#struct-441569" type="indirect">
<org type="institution" xml:id="struct-441569" status="VALID">
<idno type="ISNI">0000000122597504</idno>
<idno type="IdRef">02636817X</idno>
<orgName>Centre National de la Recherche Scientifique</orgName>
<orgName type="acronym">CNRS</orgName>
<date type="start">1939-10-19</date>
<desc>
<address>
<country key="FR"></country>
</address>
<ref type="url">http://www.cnrs.fr/</ref>
</desc>
</org>
</tutelle>
</tutelles>
</hal:affiliation>
<country>France</country>
<placeName>
<settlement type="city">Nancy</settlement>
<settlement type="city">Metz</settlement>
<region type="region" nuts="2">Grand Est</region>
<region type="old region" nuts="2">Lorraine (région)</region>
</placeName>
<orgName type="university">Université de Lorraine</orgName>
</affiliation>
</author>
<author>
<name sortKey="Badeau, Roland" sort="Badeau, Roland" uniqKey="Badeau R" first="Roland" last="Badeau">Roland Badeau</name>
<affiliation wicri:level="1">
<hal:affiliation type="laboratory" xml:id="struct-162010" status="VALID">
<orgName>Laboratoire Traitement et Communication de l'Information</orgName>
<orgName type="acronym">LTCI</orgName>
<desc>
<address>
<addrLine>46 rue Barrault F-75634 Paris Cedex 13</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.ltci.telecom-paristech.fr/</ref>
</desc>
<listRelation>
<relation active="#struct-300362" type="direct"></relation>
<relation active="#struct-302102" type="direct"></relation>
<relation name="UMR5141" active="#struct-441569" type="direct"></relation>
</listRelation>
<tutelles>
<tutelle active="#struct-300362" type="direct">
<org type="institution" xml:id="struct-300362" status="VALID">
<orgName>Télécom ParisTech</orgName>
<desc>
<address>
<addrLine>46 rue Barrault 75634 Paris Cedex 13</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.telecom-paristech.fr</ref>
</desc>
</org>
</tutelle>
<tutelle active="#struct-302102" type="direct">
<org type="institution" xml:id="struct-302102" status="VALID">
<orgName>Institut Mines-Télécom</orgName>
<desc>
<address>
<addrLine>46 rue Barrault -75634 Paris Cedex 13</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.mines-telecom.fr/</ref>
</desc>
</org>
</tutelle>
<tutelle name="UMR5141" active="#struct-441569" type="direct">
<org type="institution" xml:id="struct-441569" status="VALID">
<idno type="ISNI">0000000122597504</idno>
<idno type="IdRef">02636817X</idno>
<orgName>Centre National de la Recherche Scientifique</orgName>
<orgName type="acronym">CNRS</orgName>
<date type="start">1939-10-19</date>
<desc>
<address>
<country key="FR"></country>
</address>
<ref type="url">http://www.cnrs.fr/</ref>
</desc>
</org>
</tutelle>
</tutelles>
</hal:affiliation>
<country>France</country>
</affiliation>
</author>
</titleStmt>
<publicationStmt>
<idno type="wicri:source">HAL</idno>
<idno type="RBID">Hal:hal-01260588</idno>
<idno type="halId">hal-01260588</idno>
<idno type="halUri">https://hal.inria.fr/hal-01260588</idno>
<idno type="url">https://hal.inria.fr/hal-01260588</idno>
<date when="2016-05">2016-05</date>
<idno type="wicri:Area/Hal/Corpus">003D67</idno>
<idno type="wicri:Area/Hal/Curation">003D67</idno>
<idno type="wicri:Area/Hal/Checkpoint">000033</idno>
<idno type="wicri:explorRef" wicri:stream="Hal" wicri:step="Checkpoint">000033</idno>
<idno type="wicri:doubleKey">1558-7916:2016:Fitzgerald D:projection:based:demixing</idno>
<idno type="wicri:Area/Main/Merge">000033</idno>
<idno type="wicri:Area/Main/Curation">000033</idno>
<idno type="wicri:Area/Main/Exploration">000033</idno>
</publicationStmt>
<sourceDesc>
<biblStruct>
<analytic>
<title xml:lang="en">Projection-based demixing of spatial audio</title>
<author>
<name sortKey="Fitzgerald, Derry" sort="Fitzgerald, Derry" uniqKey="Fitzgerald D" first="Derry" last="Fitzgerald">Derry Fitzgerald</name>
<affiliation wicri:level="1">
<hal:affiliation type="laboratory" xml:id="struct-246988" status="VALID">
<orgName>NIMBUS Centre [Cork]</orgName>
<orgName type="acronym">NIMBUS</orgName>
<desc>
<address>
<addrLine>Cork Institute of Technology Campus, Bishopstown, Cork</addrLine>
<country key="IE"></country>
</address>
<ref type="url">http://nimbus.cit.ie</ref>
</desc>
<listRelation>
<relation active="#struct-313560" type="direct"></relation>
</listRelation>
<tutelles>
<tutelle active="#struct-313560" type="direct">
<org type="institution" xml:id="struct-313560" status="INCOMING">
<orgName>Cork Institute of Technology</orgName>
<desc>
<address>
<country key="FR"></country>
</address>
</desc>
</org>
</tutelle>
</tutelles>
</hal:affiliation>
<country>Irlande (pays)</country>
</affiliation>
</author>
<author>
<name sortKey="Liutkus, Antoine" sort="Liutkus, Antoine" uniqKey="Liutkus A" first="Antoine" last="Liutkus">Antoine Liutkus</name>
<affiliation wicri:level="1">
<hal:affiliation type="researchteam" xml:id="struct-420403" status="VALID">
<idno type="RNSR">201421147E</idno>
<orgName>Speech Modeling for Facilitating Oral-Based Communication</orgName>
<orgName type="acronym">MULTISPEECH</orgName>
<desc>
<address>
<country key="FR"></country>
</address>
<ref type="url">http://www.inria.fr/equipes/multispeech</ref>
</desc>
<listRelation>
<relation active="#struct-129671" type="direct"></relation>
<relation active="#struct-300009" type="indirect"></relation>
<relation active="#struct-423086" type="direct"></relation>
<relation active="#struct-206040" type="indirect"></relation>
<relation active="#struct-413289" type="indirect"></relation>
<relation name="UMR7503" active="#struct-441569" type="indirect"></relation>
</listRelation>
<tutelles>
<tutelle active="#struct-129671" type="direct">
<org type="laboratory" xml:id="struct-129671" status="VALID">
<idno type="RNSR">198618246Y</idno>
<orgName>INRIA Nancy - Grand Est</orgName>
<desc>
<address>
<addrLine>615 rue du Jardin Botanique 54600 Villers-lès-Nancy</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.inria.fr/nancy</ref>
</desc>
<listRelation>
<relation active="#struct-300009" type="direct"></relation>
</listRelation>
</org>
</tutelle>
<tutelle active="#struct-300009" type="indirect">
<org type="institution" xml:id="struct-300009" status="VALID">
<orgName>Institut National de Recherche en Informatique et en Automatique</orgName>
<orgName type="acronym">Inria</orgName>
<desc>
<address>
<addrLine>Domaine de VoluceauRocquencourt - BP 10578153 Le Chesnay Cedex</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.inria.fr/en/</ref>
</desc>
</org>
</tutelle>
<tutelle active="#struct-423086" type="direct">
<org type="department" xml:id="struct-423086" status="VALID">
<orgName>Department of Natural Language Processing & Knowledge Discovery</orgName>
<orgName type="acronym">LORIA - NLPKD</orgName>
<desc>
<address>
<country key="FR"></country>
</address>
<ref type="url">http://www.loria.fr/la-recherche-en/departements/Knowledge-and-Language-Management</ref>
</desc>
<listRelation>
<relation active="#struct-206040" type="direct"></relation>
<relation active="#struct-300009" type="indirect"></relation>
<relation active="#struct-413289" type="indirect"></relation>
<relation name="UMR7503" active="#struct-441569" type="indirect"></relation>
</listRelation>
</org>
</tutelle>
<tutelle active="#struct-206040" type="indirect">
<org type="laboratory" xml:id="struct-206040" status="VALID">
<idno type="IdRef">067077927</idno>
<idno type="RNSR">198912571S</idno>
<idno type="IdUnivLorraine">[UL]RSI--</idno>
<orgName>Laboratoire Lorrain de Recherche en Informatique et ses Applications</orgName>
<orgName type="acronym">LORIA</orgName>
<date type="start">2012-01-01</date>
<desc>
<address>
<addrLine>Campus Scientifique BP 239 54506 Vandoeuvre-lès-Nancy Cedex</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.loria.fr</ref>
</desc>
<listRelation>
<relation active="#struct-300009" type="direct"></relation>
<relation active="#struct-413289" type="direct"></relation>
<relation name="UMR7503" active="#struct-441569" type="direct"></relation>
</listRelation>
</org>
</tutelle>
<tutelle active="#struct-413289" type="indirect">
<org type="institution" xml:id="struct-413289" status="VALID">
<idno type="IdRef">157040569</idno>
<idno type="IdUnivLorraine">[UL]100--</idno>
<orgName>Université de Lorraine</orgName>
<orgName type="acronym">UL</orgName>
<date type="start">2012-01-01</date>
<desc>
<address>
<addrLine>34 cours Léopold - CS 25233 - 54052 Nancy cedex</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.univ-lorraine.fr/</ref>
</desc>
</org>
</tutelle>
<tutelle name="UMR7503" active="#struct-441569" type="indirect">
<org type="institution" xml:id="struct-441569" status="VALID">
<idno type="ISNI">0000000122597504</idno>
<idno type="IdRef">02636817X</idno>
<orgName>Centre National de la Recherche Scientifique</orgName>
<orgName type="acronym">CNRS</orgName>
<date type="start">1939-10-19</date>
<desc>
<address>
<country key="FR"></country>
</address>
<ref type="url">http://www.cnrs.fr/</ref>
</desc>
</org>
</tutelle>
</tutelles>
</hal:affiliation>
<country>France</country>
<placeName>
<settlement type="city">Nancy</settlement>
<settlement type="city">Metz</settlement>
<region type="region" nuts="2">Grand Est</region>
<region type="old region" nuts="2">Lorraine (région)</region>
</placeName>
<orgName type="university">Université de Lorraine</orgName>
</affiliation>
</author>
<author>
<name sortKey="Badeau, Roland" sort="Badeau, Roland" uniqKey="Badeau R" first="Roland" last="Badeau">Roland Badeau</name>
<affiliation wicri:level="1">
<hal:affiliation type="laboratory" xml:id="struct-162010" status="VALID">
<orgName>Laboratoire Traitement et Communication de l'Information</orgName>
<orgName type="acronym">LTCI</orgName>
<desc>
<address>
<addrLine>46 rue Barrault F-75634 Paris Cedex 13</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.ltci.telecom-paristech.fr/</ref>
</desc>
<listRelation>
<relation active="#struct-300362" type="direct"></relation>
<relation active="#struct-302102" type="direct"></relation>
<relation name="UMR5141" active="#struct-441569" type="direct"></relation>
</listRelation>
<tutelles>
<tutelle active="#struct-300362" type="direct">
<org type="institution" xml:id="struct-300362" status="VALID">
<orgName>Télécom ParisTech</orgName>
<desc>
<address>
<addrLine>46 rue Barrault 75634 Paris Cedex 13</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.telecom-paristech.fr</ref>
</desc>
</org>
</tutelle>
<tutelle active="#struct-302102" type="direct">
<org type="institution" xml:id="struct-302102" status="VALID">
<orgName>Institut Mines-Télécom</orgName>
<desc>
<address>
<addrLine>46 rue Barrault -75634 Paris Cedex 13</addrLine>
<country key="FR"></country>
</address>
<ref type="url">http://www.mines-telecom.fr/</ref>
</desc>
</org>
</tutelle>
<tutelle name="UMR5141" active="#struct-441569" type="direct">
<org type="institution" xml:id="struct-441569" status="VALID">
<idno type="ISNI">0000000122597504</idno>
<idno type="IdRef">02636817X</idno>
<orgName>Centre National de la Recherche Scientifique</orgName>
<orgName type="acronym">CNRS</orgName>
<date type="start">1939-10-19</date>
<desc>
<address>
<country key="FR"></country>
</address>
<ref type="url">http://www.cnrs.fr/</ref>
</desc>
</org>
</tutelle>
</tutelles>
</hal:affiliation>
<country>France</country>
</affiliation>
</author>
</analytic>
<series>
<title level="j">IEEE Transactions on Audio, Speech and Language Processing</title>
<idno type="ISSN">1558-7916</idno>
<imprint>
<date type="datePub">2016-05</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc>
<textClass>
<keywords scheme="mix" xml:lang="en">
<term>music processing</term>
<term>musical source separation</term>
<term>non-negative matrix factorization</term>
<term>probabilistic models</term>
<term>source separation</term>
</keywords>
</textClass>
</profileDesc>
</teiHeader>
<front>
<div type="abstract" xml:lang="en">We propose a method to unmix multichannel audio signals into their different constitutive spatial objects. To achievethis, we characterize an audio object through both a spatial and a spectro-temporal modelling. The particularity of the spatialmodel we pick is that it neither assumes an object has only one underlying source point, nor does it attempt to model the complexroom acoustics. Instead, it focuses on a listener perspective, and takes each object as the superposition of many contributionswith different incoming directions and inter-channel delays. Our spectro-temporal probabilistic model is based on the recentlyproposed α-harmonisable processes, which are adequate for signals with large dynamics, such as audio. Then, the mainoriginality of this work is to provide a new way to estimate and exploit inter-channel dependences of an object for the purposeof demixing. In the Gaussian α = 2 case, previous research focused on covariance structures. This approach is no longervalid for α < 2 where covariances are not defined. Instead, we show how simple linear combinations of the mixture channelscan be used to learn the model parameters, and the method we propose consists in pooling the estimates based on manyprojections to correctly account for the original multichannel audio. Intuitively, each such downmix of the mixture provides anew perspective where some objects are cancelled or enhanced. Finally, we also explain how to recover the different spatial audioobjects when all parameters have been computed. Performance of the method is illustrated on the separation of stereophonic musicsignals. Index Terms—source separation, probabilistic models, non-negative matrix factorization, musical source separation</div>
</front>
</TEI>
<affiliations>
<list>
<country>
<li>France</li>
<li>Irlande (pays)</li>
</country>
<region>
<li>Grand Est</li>
<li>Lorraine (région)</li>
</region>
<settlement>
<li>Metz</li>
<li>Nancy</li>
</settlement>
<orgName>
<li>Université de Lorraine</li>
</orgName>
</list>
<tree>
<country name="Irlande (pays)">
<noRegion>
<name sortKey="Fitzgerald, Derry" sort="Fitzgerald, Derry" uniqKey="Fitzgerald D" first="Derry" last="Fitzgerald">Derry Fitzgerald</name>
</noRegion>
</country>
<country name="France">
<region name="Grand Est">
<name sortKey="Liutkus, Antoine" sort="Liutkus, Antoine" uniqKey="Liutkus A" first="Antoine" last="Liutkus">Antoine Liutkus</name>
</region>
<name sortKey="Badeau, Roland" sort="Badeau, Roland" uniqKey="Badeau R" first="Roland" last="Badeau">Roland Badeau</name>
</country>
</tree>
</affiliations>
</record>

Pour manipuler ce document sous Unix (Dilib)

EXPLOR_STEP=$WICRI_ROOT/Wicri/Lorraine/explor/InforLorV4/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000033 | SxmlIndent | more

Ou

HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000033 | SxmlIndent | more

Pour mettre un lien sur cette page dans le réseau Wicri

{{Explor lien
   |wiki=    Wicri/Lorraine
   |area=    InforLorV4
   |flux=    Main
   |étape=   Exploration
   |type=    RBID
   |clé=     Hal:hal-01260588
   |texte=   Projection-based demixing of spatial audio
}}

Wicri

This area was generated with Dilib version V0.6.33.
Data generation: Mon Jun 10 21:56:28 2019. Site generation: Fri Feb 25 15:29:27 2022